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Software  Acquisition:  A  Comparison  of  DoD  and 
Commercial  Practices 


Abstract:  This  paper  will  compare  best  commercial  practice  with  the  current 
Department  of  Defense  (DoD)  processes  for  acquiring  software  and  to 
recommend  some  steps  that  can  be  taken  to  streamline  DoD  software 
acquisitions  to  minimize  overall  life-cycle  costs. 


Background 

Defining  the  Issues 

Two  issues  arise  when  discussing  commercial  practice  in  acquiring  software.  The  first  con¬ 
cerns  methods  used  by  industry  to  acquire  software  systems  similar  to  those  developed  by  ttie 
Department  of  Defense.  Second  is  the  use  of  commercial  off-the-shelf  (COTS)  products  to 
build  these  large  software  systems.  Each  of  these  issues  brings  benefits  and  risks  to  the  DoD; 
and,  while  use  of  COTS  is  worthy  of  a  major  study  itself,  this  paper  will  focus  on  the  use  of 
commercial  acquisition  methods  and  will  discuss  the  use  of  COTS  in  the  framework  of  these 
commercial  methods. 

Types  of  Software 

Both  the  DoD  and  industry  acquire  and  maintain  three  major  types  of  software  systems  that 
have  the  following  characteristics: 

1 .  Real-time  embedded  control  systems 

•  Interrupt-driven 

•  Large  numerical  processing  requirements 

•  Small  databases 

•  Tight  real-time  constraints  (miaoseconds  to  seconds) 

•  Relatively  well-defined  but  diverse  user  interfaces 

•  Requirements  and  design  driven  by  performance  constraints 

•  Examples:  Aircraft  control  system,  steel  processing  control  system 

2.  Information  systems 

•  Transaction-based 

•  Moderate  numerical  processing  requirement 

•  Large  databases 

•  Relatively  flexible  time  constraints  (seconds  to  hours) 

•  Flexible,  complex  user  interfaces 

•  Requirements  and  design  driven  by  user  interface — must  match  way  of  doing 
business 

•  Examples:  Accounting,  personnel,  and  supply  management  systems 
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3.  Command,  control,  communication,  and  intelligence  (C3I)  systems 

•  Large  numerical  processing  requirements 

•  Large  databases 

•  Near  real-time  requirements  (milliseconds  to  minutes) 

•  Flexible,  complex,  and  diverse  user  interfaces 

•  Requirements  and  des^n  driven  by  both  performance  and  interfar^ 

•  Examples:  Missile  warning  and  control  system,  telephone  switching  system, 
manufacturing,  package  delivery 

in  the  first  two  domains,  there  are  numerous  systems  in  existence  or  under  development  by 
both  the  DoD  and  irKiustry  using  a  wide  range  of  acquisition  techniques.  Large  C3I  systems, 
however,  have  fairly  limited  applications  in  industry;  but.  as  mentioned  later,  they  will  become 
more  common  in  the  future.  In  addition  to  these  domains,  commercial  vendors  produce  tools 
and  general  purpose  software  such  as  operating  systems,  word  processors,  arxj  spread¬ 
sheets. 

Acquisition  Methods 

As  the  DoD  is  looking  for  ways  to  improve  its  acquisition  methods,  much  attention  has  been 
given  to  commercial  methods.  The  focus  has  been  on  hardware  development  arxJ  manufac¬ 
turing,  but  similar  comparisons  between  DoD  and  industrial  acquisition  methods  may  be  used 
to  improve  DoD  software  acquisition.  The  following  tables  contrast  best  commercial  practice 
with  that  used  in  a  conventional  DoD  program.  Note,  however,  that  there  are  limited  cases  of 
DoD  application  of  some  of  these  commercial  practices  (the  Air  Force  PRISM  and  the  Army 
Common  Hardware  Softwar6-2  programs  are  examples)  and  that  the  practices  listed  below  do 
not  reflect  all  situations.  Note  that  DoD  separates  information  system  acquisition  from  mission- 
critical  applications  and  employs  different  regulatory  environments  for  different  domains. 

The  tables  cover  the  areas  of  requirements  definition,  vendor  selection,  development  process, 
business  practices,  integration,  testing,  delivery,  maintenance,  and  rights  in  data.  They  briefly 
describe  some  of  the  aspects  of  best  commercial  versus  DoD  practice  in  each  area.  Although 
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not  exhausting,  the  comparisons  serve  to  identify  aspects  of  software  acquisition  in  which 
large  differences  exist  between  the  two  processes.  In  general,  commercial  practices  are 
geared  so  that  systems  are  delivered  more  quickly  and  maintained  at  less  cost. 


Comparison  of  Software  Acquisition  Methods 


1  Requirements  Definition 

Best  Commercial  Practice 

Current  DoD  Practice 

Requirements  based  on  strategic  plan  and 
market  analysis. 

Requirements  based  on  using  command 

Mission  Need  Statement. 

Requirements  based  on  life-cycle  resource 
constraints. 

Requirements  based  largely  on  annual  budget 
resource  constraints. 

Detailed  requirements  generated  by 
interdisciplinary  team  including  users,  domain 
experts,  and  system  engineers. 

Detailed  requirements  generated  by  buyer  in 
collaboration  with  user.  Team  generally 
includes  domain  experts  and  acquisition 
personnel. 

Functional  specification  is  modified  by 
knowledge  of  availability  of  existing  products. 

Functional  and/or  performance  specification; 
little  to  no  regard  for  existing  products. 

Vendors  involved  early  in  study,  analysis  and 
prototyping  with  emphasis  on  reuse  and 
evolution  of  existing  systems. 

May  contract  for  prototypes,  but  contractor 
involvement  in  pre-award  discussions  is 
discouraged. 

Level  of  documentation  is  negotiable  based  on 
individual  user  needs  and  complexity  of  system 
being  developed. 

Extensive  (often  redundant  or  unnecessary) 
documentation  required  under  21 67 A. 

Tailoring  of  documentation  requirements  is 
often  minimal  or  discouraged. 

More  requirements  tradeoff  decisions 
(involving  complexity  and  schedule)  for 
reduced  time  to  field. 

Very  little  flexibility  to  trade  off  requirements 
creep  versus  complexity  and  schedule. 

Tools  used  to  create  system  models  for  use  in 
requirements  definition;  e.g.,  GUI  building. 

Requirements  definition  based  on  Mission 

Need  Statement. 

Summary 

Commercial  is  more  flexible  and  open  between  users  and  suppliers,  and  requirements  are  based  on  a 
strategic  plan.  In  the  commercial  world,  there  is  more  willingness  to  adjust  requirements  based  on  availability 
of  products  and  thus  to  field  a  system  sooner  and  evolve  it  to  include  more  capability. 
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1^  Vendors 

Selection 

Best  Commercial  Practice 

Current  OoO  Practice 

Solicit  muKipIo  (but  not  all)  qualified  vendors. 
Encourage  teaming  with  a  view  to  attaining  a 
relationship  that  covers  the  entire  life  cycle  and 
fosters  tradeoffs  in  cost  and  schedule. 

Solicit  all  possible  vendors.  Vendor  proposals 
must  meet  100%  of  requirements.  Teaming 
seldom  encouraged.;  development  and 
maintenance  usually  separate  entities. 

Compare  vendor  history  and  experience.  Maintain 
long-term  relationships. 

Can  compare  previous  performance,  but  normally 
cani  have  tong-term  relationships. 

The  organization  that  will  be  responsible  for  a 
system  over  its  full  life  cycle  is  heavily  involved 
from  the  beginning. 

Maintenance  organization  not  usually  involved  in 
vendor  selection  process. 

Use  site  visits  and  demonstrations  to  gain 
knowledge  of  vendor  capabilities. 

Site  visit  only  by  capability  evaluation  team,  or 
other  expert  teams.  Visits  are  very  structured. 

Overall  goals;  (1 )  obtain  product  at  reasonable 
cost  as  soon  as  possible;  and  (2)  achieve  the 
business  case  for  the  system. 

Overall  goal:  Obtain  lowest  cost  product  that 
rigorously  meets  all  requirements,  but  be  fair. 

Relatively  few  review  and  approval  steps  once 
vendor  is  selected. 

Review  and  approval  process  more  structured 
and  complex  once  vendor  selected. 

Past  performance  weighted  heavily  (sometimes 
primary  factor)  in  selection  process. 

Past  performance  considered,  but  usually  only  as 
a  minor  factor. 

Mora  flexibility  in  vendor  selection  based  on 
metrics  and  overall  assessment. 

Selection  of  vendor  forced  by  use  of  predefined 
metrics  for  proposal  evaluation. 

Summary 

Very  different  processes  with  commercial  much  more  flexible  but  with  no  requirement  for  fairness,  or  to  maintain 
the  public  trust.  Commercial  encourages  vendors  to  offer  best  solution,  but  solution  may  not  meet  1 00%  of  the 
requirements.  Teaming  and  long-term  relationships  are  more  easily  accommodated  by  industry. 
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Development  Process 

Best  Commercial  Practice 

Current  DoD  Practice 

Vendor  often  tailors  existing  systems  and  uses 

COTS.  System  designed  to  fit  in  a  defined 
architecture  or  product  line. 

Vanes  with  application.  Some  systems  use 

COTS.  However,  usually  a  new  system  that 
doesn't  reuse  legacy  software.  Unique  systems 
are  built  with  little  regard  for  architecture. 

Buyer  may  have  heavy  involvement  in  design  and 
development  as  part  of  the  team  (Integrated 

Product  Development  team) 

Formal,  structured  spiral,  or  waterfall  model. 

Buyer  oversees,  but  team  approach  is  not  usually 
emphasized. 

Reviews  typically  informal  and  stress  progress 
against  goals. 

Reviews  usually  vary  formal  and  include  technical 
design  details  in  addition  to  progress  metrics. 

Heavy  user  involvement. 

Limited  user  involvement. 

Heavy  buyer  involvement. 

Vendor  embraces  one  or  more  industry  standards 
which  improves  interface  and  integration  with 

COTS  products. 

Government  and  industry  standards  called  out. 

Not  all  government  standards  enhanced  by  COTS 
products. 

Buyer  requirements  may  be  translated  to  more 
“general  purpose*  requirements  for  potential 
software  reuse. 

Tsulored  system;  littia,  if  any,  focus  on  designing  in 
reusable  code. 

Management  reviews  and  degree  of  oversight  are 
commensurate  with  size  and  risk  of  program. 

Notably  more  detailed  reviews  and  oversight 
performed. 

Prototyping  common,  with  joint  applications 
development  teams  (user  and  developer)  working 
to  clarify  requirements  and  incorporate  new 
requirements  that  do  not  affect  cost  or  schedule. 

Prototyping  seldom  used,  but  becoming  more 
popular. 

Summary 

Commercial  more  flexible  with  likelihood  of  a  team  approach  and  is  biased  toward  reuse  and  tailoring  of  existing 
systems.  Product  improvements  are  anticipated. 
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Business 

Practices 

Best  Commercial  Practice 

Current  DoO  Practice 

Informal  contracting,  joint  ventures,  partnerships 
with  mutual  economic  benefit  and  vested  interest 
in  success. 

Formal  contract  with  little  motivation  to  reduce 
cost. 

Oversight  built  on  established  relationships. 

Burdensome  cost  accounting  procedures 
required;  extensive  oversight,  reporting,  and 
documentation  requirements. 

Can  hire  and  fire  vendors  and  managers. 

Government  personnel  regulations,  policies,  arrd 
practices  determine  qualifications  of  its 
managers,  rotations  of  assignment,  and  training. 

Multi-year  effort  and  funding. 

Multi-year  effort.  Yearly,  unpredictabla  funding. 

Summary 

Commercial  practice  more  flexible  with  greater  incentives. 

Integration  Testing  a.nd  Delivery 

Best  Commercial  Practice 

Current  DoO  Practice 

Unless  system  is  for  a  new  plant,  then  there  are 
major  “cut-over*  issues. 

Similar  “cut-over*  or  transition  issues 

Sometimes  difficult  to  assemble  complete  system 
in  labora'~'’'y  environrr^ent  due  to  cost.  Testing 
usually  done  in  client's  facility. 

Usually  integrate  system  in  laboratory  prior  to 
operational  testing. 

Development  testing  vs.  operational  testing  via 
statutory  mandate. 

Beta  testing  widely  used  to  quickly  find  errors. 

Little  beta  tasting. 

unimate  acceptance  authority  rests  with  buyer, 
not  a  separate  organization. 

Structured,  specified  operational  testing 
conducted  by  separate  authority.  Acceptance 
authority  rests  with  buyer. 

Summary 

Integration  and  functional  testing  seem  appropriate  to  the  need.  DoD  use  of  separate  test  agency  adds  time  and 
complexity. 
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Maintenance 


Best  Commercial  Practice 

Current  DoD  Practice 

Organic  support  shifting  to  outsourcing  or 
vendor. 

Organic  support,  with  reluctance  to  be 
dependent  on  vendor.  Use  of  depot 
maintenance  makes  interoperability  issues 
more  manageable.  Also,  must  be  responsive  to 
user  for  critical  systems. 

Summary 

The  DoD  and  industry  have  different  requirements  and  must  be  careful  when  selecting  a  maintenance  strategy 
appropriate  to  their  needs. 

Rights 

i.n  Data 

Best  Commercial  Practice 

Current  DoD  Practice 

If  custom  development,  buyer  gets  all  rights,  but 
vendor  may  retain  right  to  subsequent  sales. 

Specified  by  contract. 

Government  usually  demands  all  rights  for 
government  use  due  to  organic  support  and 
maintenance  needs,  and  competition  (via 
statutory  mandate). 

If  tailored  version  of  standard  system,  buyer  only 
gets  rights  to  tailored  parts. 

Same  as  commercial 

May  have  exceptions  for  proprietary  material 

Summary 

Similar,  but  commercial  is  a  little  more  flexible  especially  regarding  resales. 

Observations 

As  described  in  the  table,  some  commercial  acquisition  practices  are  significantly  different 
than  those  used  by  most  DoD  programs.  Some  of  these  appear  to  be  more  efficient.  Some 
would  require  changes  to  federal  law  and  the  Federal  Acquisition  Regulations  (FAR),  but  most 
can  be  adopted  more  easily  by  changing  aojulsition  practices.  However,  the  current  mindset 
of  the  DoD  acquisition  process  leads  to  a  very  conservative  approach  versus  a  more  flexible 
and  aggressive  process.  The  DoD  culture  tends  to  prevent  government  program  managers 
from  taking  advantage  of  even  relatively  simple  changes  to  current  practice.  Some  activities 
are  now  underway  in  the  Defense  Acquisition  Pilot  Program  to  gain  relief  from  federal  law  and 
FAR  requirements.  Specific  actions  taken  by  each  program  director  should  be  collected  along 
with  lessons  learned  as  the  acquisitions  proceed. 
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The  following  section  highlights  a  few  of  the  more  significant  practices  in  the  requirements, 
source  selection  and  development  phases  and  discusses  some  regulatory  aspects  of  chang¬ 
ing  DoD  practice.  It  should  be  noted,  however,  that  the  DoO  should  not  simply  copy  attractive 
commercial  practices.  Rather,  the  practice  must  be  analyzed  and  possibly  modified  to  take  ad¬ 
vantage  of  the  most  efficient  aspects  that  would  apply  to  the  specific  DoD  application.  Since 
DoD  program  managers  must  maintain  the  public  trust,  care  must  be  taken  when  applying  new 
acquisition  techniques. 

Requirements 

The  largest  differences  between  commercial  and  DoD  practices  lie  in  the  user-buyer- 
developer  relationship.  Industry  considers  the  availability  of  existing  products  in  this  phase  and 
is  more  willing  than  the  DoD  to  trade  functionality  with  availability  to  decrease  cost  and  sched¬ 
ule.  Systems  are  thus  delivered  earlier  and  are  then  evolved  to  include  later  requirements.  In 
addition,  the  best  commercial  practice  is  when  an  integrated  product  development  (IPD)  team, 
including  suppliers,  is  formed  eariy  and  is  kept  together  throughout  the  program  lifetime.  Also, 
companies  that  provide  software  think  in  terms  of  product  lines  that  fit  into  standard  architec¬ 
tures  with  tailorable  products.  Service  acquisition  professionals  could  set  up  test  acquisitions 
that  take  advantage  of  the  functionaiity/cost  tradeoffs  without  modification  of  the  FAR.  They 
can  also  organize  with  a  team  approach.  By  tailoring  DoD  STD  21 67a.  unnecessary  documen¬ 
tation  and  accounting  overhead  could  be  saved.  A  study  by  Air  Force  Electronic  Systems  Cen¬ 
ter  indicates  that  documentation  requirements  can  be  reduced  by  over  60%  for  mature 
developers.  In  addition,  use  of  IPD  teams  gives  ^e  government  visibility  into  programs  that 
cannot  be  obtained  through  21 67a  documentation  alone. 

Vendor  Selection 

Industry  and  the  DoD  have  significantly  different  practices  here.  Since  industry  has  a  funda- 
mentaliy  different  relationship  with  suppliers,  major  changes  to  the  FAR  would  be  required. 
The  ability  to  negotiate  with  suppliers  and  to  engage  in  long  term  contractual  relationships 
which  cover  both  the  development  and  maintenance  phases  is,  so  far,  forbidden  by  regula¬ 
tions  or  interpretations  of  regulations.  Due  to  tiie  requirement  to  maintain  public  trust  and  to 
be  rigorousiy  fair,  the  DoD  is  constrained  in  its  ability  to  radically  change  vendor  selection  tech¬ 
niques. 

Use  of  Commercial-Off-the-Shelf  (COTS)  Software 
Development 

As  some  development  agencies  are  beginning  to  pioneer,  COTS  products  can  be  used  to  build 
much  of  a  system,  especially  infrastructure  elements.  However,  use  of  COTS  is  not  without 
risk.  As  noted  in  the  SEI  appraisal  of  software  development  risk  management,  problems  can 
be  encountered  in  using  COTS  in  the  following  areas: 

•  Customizing;  Changes  to  interfaces  and  accommodation  of  version  releases  can  have 
significant  impact  on  other  parts  of  a  system. 
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•  Testability  and  integrability:  No  clear  traceability  nor  clear  line  of  responsibility. 

•  COTS  quality  versus  system  quality;  Reliat»lity  figures  typically  don’t  apply  to  software. 

Also,  COTS  performance  and  short  lifetimes  can  seriously  affect  a  large  system  development. 
When  performance  limitations  are  found  to  be  due  to  COTS  components,  alternatives  are 
sometimes  limited  to  substituting  a  different  product  that  may  have  different  interfaces  and  up¬ 
grading  the  hardware  platform.  Neither  of  these  alternatives  is  usually  cheap  nor  quick.  COTS 
products  tend  to  change  rapidly,  with  attendant  testing  and  analysis  requirements  for  system 
builders.  A  new  operating  system,  for  example,  can  take  a  team  of  people  six  months  to  ade¬ 
quately  test  and  validate  for  use  on  a  major  project.  COTS  use  also  affects  programmatic  de¬ 
cisions  concerning  maintenance  and  product  lifetimes.  The  aforementioned  volatility  and  the 
chance  that  a  vendor  will  go  out  of  business  can  affect  systems  with  expected  lifetimes  of  de¬ 
cades.  These  risks  can  be  mitigated  as  long  as  they  are  recognized.  For  example,  domain  en¬ 
gineering  allows  both  continuous  assessment  of  COTS  products  as  well  as  current  familiarity 
with  industry  standards. 

Support 

In  the  post  deployment  software  support  (PDSS)  phase,  support  agencies  must  develop  a  sus¬ 
tainment  plan  that  addresses  defect  and  enhancement  maintenance.  Defect  maintenance  of 
COTS  software  may  best  be  handled  through  a  warranty.  However,  this  should  be  negotiated 
judicially  to  ensure  total  warranty  for  all  defects.  COTS  enhancements  must  be  supported  by 
an  innovative  contract  that  defines  well  the  contractor’s  responsibility  for  delivering  new  COTS 
products  that  support  functionality  enhancement.  This  requires  a  P3I  enhancement  plan  for 
future  COTS  Insertion. 

Evolutionary  development  is  one  means  to  expand  a  system  based  on  resource  and  technol¬ 
ogy  constraints.  Evolutionary  software  maintenance  has  come  about  because  of  the  ease  of 
adding  new  functionality  into  a  system  through  software  improvements  vs.  hardware.  Howev¬ 
er,  the  insertion  of  COTS  software  places  an  added  element  of  risk  for  software  enhance¬ 
ments.  Some  questions  related  to  support  of  COTS  products  are  discussed  below: 
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1.  How  does  the  life  cycle  software  support  (LOSS)  activity  stimulate  competition  for 
COTS  that  will  ensure  a  fair  and  reasonable  cost? 

2.  How  does  the  LOSS  ensure  that  the  COTS  will  technically  integrate  into  the  system 
and  continue  to  maintain  an  open  architecture  evolutionary  development? 

3.  Reuse  has  a  very  cost  effective  roie  in  evolving  systems.  What  is  the  effect  of  COTS 
vs.  reuse?  is  there  a  process  for  reusing  COTS  for  different  system  domains,  and  what 
would  the  proprietary  issues  be? 

4.  Software  engineering  methods  and  techniques  that  are  now  becoming  practice 
emphasize  modei-based  software  architectures.  This  new  approach  to  developing 
software  produces  a  weli  defined  design  that  offers  substantial  potential  for  reuse.  Will 
COTS  support  this  cost  effective  maintenance  approach?  Is  this  currently  being  used 
in  commercial  practices?  Available  data  shows  limited  introduction  for  commerciai 
development. 

How  does  a  vendor  support  a  commercial  system?  There  are  numerous  ways  in  practice,  the 
most  common  being  the  maintenance  warranty.  Maintenance  warranties  vary  depending  on 
the  type  of  system,  and  essentially  how  much  ttie  buyer  is  willing  to  pay  for  the  support.  Gen¬ 
erally,  a  warranty  only  covers  maintenance  of  defects.  If  a  commercial  buyer  desires  to  im¬ 
prove  a  system  through  evolutionary  means,  there  is  a  costly  maintenance  contract  required 
since  most  of  the  enhancements  are  accomplished  through  reengineering. 

COTS  has  yet  to  scale  up  for  insertion  into  some  software  architectures  unless  it  has  been 
specifically  engineered  for  a  given  domain.  However,  the  engineering  approach  taken  by  the 
PRISM  project  to  build  a  reusable  COTS  product  line  for  a  specific  domain  (command  and 
control)  is  an  example  of  successful  use  of  COTS  in  one  large  domain. 

Commercial  practices  for  PDSS  tend  to  be  rather  ad-hoc.  Commerciai  systems  software  is 
supported  by  an  add-on  contract  that  generally  covers  defect  detection  and  resolution.  En¬ 
hancements  are  normally  contracted  for  at  system  project  initiation  for  the  insertion  of  new 
functionalities.  Both  commerce  and  government  have  put  more  thought  into  the  architectural 
and  acquisition  aspects  of  COTS,  but  different  approaches  to  PDSS  means  that  commercial 
experience  may  not  easily  map  to  the  government. 

Convergence  of  C3  Projects 

Industry,  with  the  move  toward  just-in-time  ordering  and  agile  manufacturing,  is  beginning  to 
experience  the  need  for  large  near  real-time  command  and  control  systems  similar  to  those 
long  used  by  the  DoD.  Indeed,  some  industries,  such  as  communications  and  manufacturing, 
have  already  developed  systems  similar  to  those  used  for  tactical  military  command  and  con¬ 
trol.  These  systems,  depending  on  the  application  domain,  consist  of  between  60%  and  80% 
infrastructure  (database,  user  interface,  etc.).  The  market  for  this  infrastructure  will  thus  grow 
from  one  customer  (the  DoD)  to  many.  The  role  of  the  DoD  in  the  future,  then,  should  be  to 
cooperate  with  industry  to  encourage  the  development  of  commercial  dual-use  products  to 
populate  this  infrastructure.  This  would  make  more  robust  technology  available  for  both  DoD 
and  industrial  systems. 
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Recommendations 

Commercial  Practice  Experiments 

The  OoD  should  encourage  and  closely  monitor  pilot  projects  that  employ  carefully  chosen 
commercial  practices  in  the  requirements  definition,  vendor  selection  and  development  phas¬ 
es  of  selected  program  acquisitions.  As  the  Defense  Science  Board  Task  Force  Report  on  En¬ 
gineering  in  the  Manufacturing  Process  points  out,  these  experiments  should  demonstrate  the 
following  benefits: 

•  Reduce  ambiguity 

•  Eliminate  delay 

•  Reduce  risks 

•  Reduce  cost 

•  Increase  quality 

•  Increase  maintainability 

•  Responsiveness 

•  Preservation  of  design/architectural  integrity 

•  Enhance  integration  with  legacy  systems 

•  Responsiveness  to  original  integration  and  to  changes 

•  Reliability  of  interfaces 


Attaining  these  benefits  in  the  software  area  should  be  goals  of  experiments  that  use  the  fol¬ 
lowing  techniques  in  the  first  three  program  phases: 

1 .  Requirements  Phase 

•  Reduce  ambiguity  by  extensive  simulation  and  prototyping.  Specify  data  model  and 
global  standards. 

•  Maintain  prototypes  as  the  baseline  throughout  the  development  to  quickly  analyze 
changing  requirements. 

•  Eliminate  delay  and  improve  quality  and  efficiency  by  specifying  standard  interfaces 
to  minimize  data  manipulation. 

•  Perform  functionality/cost  tradeoffs  eariy  in  the  requirements  phase  to  determine  if 
dramatic  time  or  cost  savings  can  be  obtained  with  relatively  small  changes  to 
requirements. 

•  Maximize  flexibility  by  stating  system  performance  and  functional  requirements  as 
broadly  as  possible,  consistent  with  supporting  the  intended  mission. 

•  Tailor  documentation  requirements  to  emphasize  those  items  needed  by  the  end 
user  to  understand,  use  and  maintain  the  final  software  product  and  minimize  the 
amount  of  (often  unused)  documentation  associated  with  recording  each  step  of  the 
design  process. 
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•  include  COTS  products  surveys  and  vendor  site  visits  as  part  of  the  requirements 
generation/request  for  prop^al  (RFP)  writing  process  to  allow  requested 
functionsriity  and  performance  requirements  to  be  adjusted  to  accommodate 
existing  software  products  when  consistent  with  mission  capabilities. 

•  Include  contract  mechanisms  for  incremental  requirements  definiticn/improvement 
activities  rather  than  trying  to  cast  all  requirements  in  concrete  at  program  go- 
ahead. 

•  Give  users  incentive  to  foltow/encourage  commercially  available  functions  and 
forego  military  service  unique  requirements. 

2.  Vendor  Selection  Phase 

•  Reduce  cost  and  risk  by  using  integrated  product  teams. 

•  Involve  vendors  early  in  the  conceptual  p^ase. 

•  Encourage  the  use  of  product  lines  in  standard  architectures. 

•  Emphasize  quality  engineering  processes. 

•  Adopt  an  acquisition  strategy  that  readily  accepts  change  to  accommodate  volatile 
requirements. 

•  Encourage  adherence  to  commercial  open  systems  standards,  rather  than 
restricting  offerers  to  compliance  with  military  standards.  This  would  be  easier  to 
accomplish  if  government  personnel  involved  in  RFP  preparation  received  training 
in  existing  and  emerging  commercial  standards. 

•  When  appropriate  and  feasible,  vendor  compliance  to  the  goals  of  the  SEI 
Capability  Maturity  Model  ought  to  be  a  heavily  weighted  selection  criterion. 

•  Include  metrics  associated  with  the  extent  to  which  COTS  is  used  to  satisfy 
requirements  as  an  explicit  part  of  the  evaluation  criteria. 

•  Use  “best  value"  procurement  techniques  to  allow  more  advantageous  tradeoffs 
between  requirement  satisfaction  and  costs. 

3.  Development  Phase 

•  Use  open  systems  standards  to  reduce  ambiguity,  reduce  cost  and  improve  quality. 

•  Eliminate  delay  by  enforcing  interface  standards  and  reducing  the  number  of 
Engineering  Change  Proposals.  Eliminate  “nice  to  haves”  and  delay  "have  to  haves" 
to  pre-planned  product  improvement  (P3I)  modifications. 

•  Reduce  risk  and  improve  product  quality  by  thoroughly  evaluating  COTS  products, 
enforcing  interface  standards  and  by  using  virtual  interfaces,  such  as  developed  by 
the  STARS  and  PRISM  programs. 

•  Consider  the  use  of  selected  commercial  software  development  practices  in  place 
of  tailoring  2167a  or  other  military  standards. 

•  Encourage  incremental  or  spiral  development  approaches  with  provision  for  hands 
on  user  evaluations  of  early  software  releases  (similar  to  the  idea  of  beta  tests  in 
the  commercial  world). 

•  Tailor  Program  Management  Review/Design  Review  agendas  to  focus  on 
programs,  plans,  and  status  rather  than  on  inappropriately  detailed  design 
presentations.  Relegate  detailed  design  oversight  activities  to  less  formal  forums, 
and  implement  via  government  membership  in  integrated  development  teams. 
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•  Give  contractors  incentive  to  commercialize  items  modified  or  developed  under  the 
contract  whenever  possible.  This  can  be  done  in  ways  which  can  ensure  that  the 
government  will  get  low  cost  upgrades  to  government  owned  systems  as  new 
commercial  versions  are  developed  and  released.  Such  arrangements  can  also 
decrease  the  government's  maintenance  support  costs. 

•  Give  incentive  to  contractors  to  use  reuse  libraries. 

•  Establish  “clearing  houses”  for  determining  what  commercial  products  are  useful  for 
acquirers. 


Suggested  Approach 

In  addition  to  the  pilot  projects,  the  government  could  apply  selected  commercial  practices  to 
an  existing  Advanced  Technology  Demonstration  (ATD)  that  has  a  significant  software  con¬ 
tent,  such  as  the  Advanced  Field  Artillery  System  Fire  Control/Battiefield  Management  Sys¬ 
tem.  ARPA  can  underwrite  the  use  of  practices  in  an  ATD  that  would  be  viewed  as  risky  in  a 
major  program  acquisition,  both  technically  and  managerially,  but  that  has  significant  potential 
payoff  in  the  way  that  the  DoD  acquires  systems. 

Infrastructure  Development 

ARPA  Should  work  with  leading  edge  industries  to  determine  command  and  control  system 
infrastructure  requirements  and  initiate  development  of  dual-use  technologies  to  populate  the 
infrastructure. 

One  way  to  accomplish  this  Is  to  encourage  the  development  of  product  lines  based  on  archi¬ 
tectures  such  as  those  defined  in  the  Domain-Specific  Software  Architectures  Project.  Tailor- 
able  products  from  these  lines  could  then  be  used  by  both  the  DoD  and  industry  to  populate 
infrastructures  of  systems  in  all  three  domains,  but  particularly  in  C3  systems  that  are  large, 
complex,  and  expensive.  One  of  the  issues  that  must  be  explored  concerns  the  tradeoffs  in 
ownership  of  architectures.  At  least  three  cases  should  be  considered; 

1 .  Ownership  by  the  government  (customer). 

2.  Ownership  by  the  community,  such  as  with  standards. 

3.  Ownership  by  the  vendor. 

It  is  important  that  infrastructure  efforts  be  cognizant  of  the  Technical  Architecture  Framework 
for  Information  Management  (TAFIM)  program.  In  particular,  the  DISA  DoD  TAFIM  Volume  2, 
published  in  June  1993,  provides  a  solid  framework  into  which  the  various  standards  used  to 
characterize  open  systems  can  be  placed. 
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